Information Theory and Allocating Resources

The world of information theory is inextricable linked to the world of gambling. Here, gambling takes on the meaning of any situation where resources are distributed among various investments with a hope of maximizing the total return. A look at how the doubling rate of a gambling scheme can be related to the amount of side information about the gamble present is discussed, as well as a discussion on how the increase of side information directly relates to an increase in the upper bound of the doubling rate. The optimal gambling scheme under “horse race” situations is shown to be Kelly gambling and algorithms for the best allocation of resources under such circumstances are discussed. The risk of a gamble can be minimized if the gambler knows something about the true distribution of outcomes. In situations where the true distribution is not known, the gambler must make an assumption on the distribution of the outcomes and pay the penalty of extra risk in the form of a Kullback-Liebler distance between the true and estimated distribution. This risk can be bound tighter by making a connection between the KL-divergence and the Hellinger distance. Finally, a discussion of information theory’s relation to investments in stock markets is presented. It is shown that the there is an optimal portfolio strategy that will maximize the doubling rate of investments. This optimal portfolio strategy is impossible to obtain without extraordinary side information, but algorithms exist to come reasonably close.